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Abstract 


The Surgical Council on Resident Education (SCORE) program was designed to create a national surgical curriculum for 
residents. This curriculum includes a list of topics to be covered in a five-year general surgery residency training program. 
A resident in-service exam (ABSITE) is administered annually to assess knowledge of applied science and management of 
clinical problems. We hypothesized that resident use of the SCORE curriculum positively correlates with ABSITE scores. All 
PGY-2, PGY-3, and PGY-4 general surgery residents for two academic cycles (July 1, 2017 through January 24, 2019) were 
included. Individual total time on the SCORE portal in minutes was collected. ABSITE scores for the same time period were 
used for comparison. Residents who scored 30th percentile or below were identified as requiring remediation. The data was 
analyzed with descriptive statistics, student’s-t test, and ANOVA with p<0.05 considered significant. There were 51 residents 
included in the study (16 PGY-2s, 17 PGY-3s, and 18 PGY-4s). There was no difference in SCORE usage (p=0.648) or ABSITE 
percentile (p=0.353) between the different PGYs. There were 20 residents that scored 30th percentile or below on the ABSITE. 
The residents that scored above the 30th percentile used SCORE significantly longer (2742 min vs. 1164 min) compared to 
residents that scored 30th percentile or below (p=0.022). Increased usage of the SCORE portal was associated with higher 
performance on the ABSITE. Further investigation is needed to determine which aspects of the portal lead to improved scores. 
This study provides additional evidence that surgery residents should use the SCORE portal to achieve appropriate surgical 


and clinical knowledge. 
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Education in Surgery. 


Introduction 


The American Board of Surgery In-Training Examination 
(ABSITE) is a multiple-choice examination administered to 
surgery residents in the United States since 1975 [1]. The 
ABSITE is offered annually and is designed to measure the 
progress attained by residents in their knowledge of applied 
science and management of clinical problems related to 
surgery. Following completion of residency, the General 
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Surgery Qualifying Exam (QE) is one of two exams taken to 
obtain board certification. ABSITE scores have been shown to 
correlate with future performance on the QE [2]. Therefore, 
ABSITE scores are of upmost importance during surgical 
residency. The ABSITE exam is standardized and considered 
reproducible [3]. Thus, surgical residency program directors 
utilize ABSITE scores as one of the measures to assess 
resident performance [4]. According to the American Board 
of Surgery, the ABSITE is furnished to program directors as a 
formative evaluation instrument to assess resident progress 
[5]. Program directors of surgical residencies therefore utilize 
ABSITE scores to evaluate clinical competency, medical 
knowledge, overall progress, and appropriate promotion 
within the residency program [4]. 
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The Surgical Council on Resident Education (SCORE) 
was established in 2004 and is a consortium of seven US 
based surgical organizations [6]. The goal of SCORE was 
to develop a standardized, competency-based curriculum 
to provide residents and residency programs high-quality 
educational materials structured for self-directed learning. 
The SCORE portal, a web-based online platform, was created 
in 2009 and made available by purchased subscription. By 
2012, 96 percent of general surgery residency programs 
subscribed to the SCORE web portal to support resident 
education [6]. Early analysis of initial subscriptions to 
the portal showed considerable improvement in mean 
American Board of Surgery Qualifying Examination scores 
for residents in programs with subscriptions to SCORE [7]. 
In 2014, the ABSITE was aligned with the SCORE Curriculum 
Outline for General Surgery, with ABSITE content directly 
corresponding to that of SCORE [8]. Overall, approximately 
72% of the ABSITE questions address SCORE Patient Care 
topics while 24% cover SCORE Medical Knowledge topics. 
National resident use of SCORE has been described and 
the ABSITE is now a prominent factor stimulating SCORE 
usage [9-11]. Other institutions have shown that time spent 
on electronic-based resources, including SCORE, predict 
future ABSITE performance [12]. With extrapolation, use of 
SCORE may therefore predict future performance on the QE. 
We hypothesized that surgery resident SCORE portal time 
investment would correlate positively with ABSITE scores. 


Methods 


This was a single institution, retrospective review of all 
postgraduate (PG) year 2, PGY-3, and PGY-4 general surgery 
residents for two academic cycles (July 1, 2017 through 


January 24, 2019) in a large, urban based training program. 
Residents were reviewed individually during each academic 
cycle. The first academic cycle began July 1, 2017 and ended 
with completion of the ABSITE. The second cycle began July 
1, 2018 and similarly ended with completion of the ABSITE. 
All residents were encouraged to use the SCORE portal by the 
program director. Each resident was provided an individual 
logon for the SCORE portal. Residents were informed that 
time investment on the portal would be tracked and recorded. 


The SCORE portal provided the number of times and the 
minutes spent during each logon for the included residents. 
Sessions extending over 180 minutes were excluded from 
the analysis and total time used in minutes was calculated 
for each resident per academic cycle. ABSITE percentiles for 
the same time period were used for comparison. Residents 
who scored in the 30th percentile or below were identified 
as requiring formal remediation. The data was analyzed with 
descriptive statistics, student’s-t test, and ANOVA. A p-value 
<0.05 was considered significant. 


Results 


There were 51 residents included in the study between 
July 1, 2017 and January 24, 2019. Of the 51 residents, 16 
(31%) were PGY-2s, 17 (33%) were PGY-3s, and 18 (35%) 
were PGY-4s. Overall, the average time spent using the 
SCORE portal for all included residents was 2124 minutes. 
The average time spent using SCORE per academic cycle was 
1780 minutes for PGY-2s, 2557 minutes for PGY-3s, and 2021 
minutes for PGY-4s. There was no significant difference in 
SCORE usage (p=0.648) between the different PGYs (Table 
1). 


N Time(min) ABSITE percentile 
PGY-2 16 1780 37 
PGY-3 17 2557 51 
PGY-4 18 2021 48 
P=0.648 P=0.353 


Table 1: The average ABSITE percentile for all residents included was the 45th percentile. 


The average percentile was 37th for the PGY-2s, 51st for 
the PGY-3s, and 48th for the PGY-4s. There was no significant 


difference in the ABSITE percentile score (p=0.353) between 
the different postgraduate years (Table 2). 


ABSITE percentile 30or Less 


ABSITE percentile 31o0r More 


Number of Residents 20 


SCORE Usage 


1167minutes 


31 
2742minutes 


P=-0.022 


Table 2: There were 31 (61%) residents that scored in the 31st percentile or above on the ABSITE. The residents that scored 
above the 30th percentile used SCORE significantly longer (2742 min vs. 1164 min) compared to residents that scored 30th 


percentile or below (p=0.022). 
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Discussion 


Increased use of the SCORE portal was associated with 
higher performance on the ABSITE. The SCORE portal has 
become an ubiquitous learning tool among general surgery 
residents in the United States, with specific reliance towards 
ABSITE preparation [10]. The vast majority of surgical 
residencies carry subscriptions to the portal and therefore 
allow residents to access the curriculum to promote self- 
directed study. Many surgical residency programs have 
also formally incorporated components of the portal into 
structured education. Within our program, the importance 
of SCORE is emphasized and certain milestones within the 
curriculum are now annually required. 


With the ability to track individual SCORE usage and 
completion of modules, we have the ability to ensure what 
we deem as appropriate time spent on the curriculum. This 
Week in Score (TWIS) is series of quizzes for specific topics on 
the SCORE portal. Our program monitors TWIS completion 
and quiz scores as a component of the Clinical Competency 
Committee (CCC) assessment of residents. The CCC is an 
essential component of resident evaluation required for 
program accreditation. Our intent is to provide adequate 
and comprehensive knowledge which should be reflected 
in ABSITE scores given the similar topics and learning 
objectives. Ultimately, the goal is to see this translated to the 
General Surgery Qualifying Exam and subsequent Certifying 
Exam (CE) for board certification. 


While mastering surgical technique and achieving 
clinical competency are essential goals for a graduating 
surgical resident, of equal importance is obtaining board 
certification by passing the QE. This exam is administered 
by The American Board of Surgery and is designed to assess 
a surgeon’s knowledge of general surgical principles and 
applied science [5]. The content comprised by the QE is 
considered necessary for the practicing general surgeon 
and a vital component for success. The correlation between 
ABSITE performance and subsequent pass rates of the QE is 
well-known and established, which reinforces the importance 
of achieving a high ABSITE score [2,13]. A multi-institutional 
study by de Virgilio et al. demonstrated that ABSITE scores 
of less than the 35th percentile at any time during residency 
was associated with failure of the QE [14]. Another study 
by the same authors showed ABSITE scores of less than the 
30th percentile once, or less than the 35th percentile more 
than once during residency is correlated with failure of the 
QE [15]. An even lower score on the ABSITE of less than the 
25th percentile was associated with failure of both the QE 
and the CE [14]. 


Because ABSITE exam questions are mapped to the 
SCORE curriculum, one would expect that greater time on the 
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SCORE portal improves ABSITE scores. We are not the first 
authors to describe this relationship. Hancock et al. identified 
high scoring residents (>80th percentile) spent significantly 
more minutes on SCORE per day [12]. Perez et al. conducted 
a single institution retrospective review of general surgery 
resident use of two electronic study resources, including 
SCORE and TrueLearn [12]. TrueLearn is an independent 
company that provides question banks to training physicians 
for preparation of licensure exams in medicine. Distinct 
differences in time spent on the electronic study resources 
were identified in residents with higher ABSITE scores. 
Longer online sessions (15 minutes per day or 35 minutes 
per session) and increasing use prior to the ABSITE exam 
was seen in the higher scoring residents. Interestingly, high 
ABSITE performers were more likely to access SCORE later 
at night and on weekends, generally outside of regular work 
hours [12]. 


Because of the importance of the QE and the correlation 
with ABSITE, it is necessary to identify residents at potential 
risk for failure. Our institution has adopted an ABSITE 
remediation program called Accelerated Clinical Education 
in Surgery (ACES) for residents scoring below the 30th 
percentile, as this is consistent with a higher QE failure rate 
[16]. This program was initially developed prior to the SCORE 
portal and instead utilized weekly reading and quizzes 
from the Surgical Education and Self-Assessment Program 
(SESAP) and other sources. With the development of SCORE, 
we have maintained our ACES program but have restructured 
it with SCORE modules and SCORE TWIS questions. We have 
observed more significant improvements in ABSITE scores 
of remediating residents following the adoption of SCORE. 
In our remediation experience, the improvement in ABSITE 
score is greatest in those residents who spend longer times 
on the SCORE portal and use it throughout the academic year. 


This discussion raises questions of exactly how residents 
prioritize study topics and also how free time is designated 
for education. There have been limited studies evaluating 
precisely what is studied and when. Regarding free time, 
residents theoretically have more time for education in the 
context of a restricted 80-hour work week. Studies have 
shown that regular study habits, good conference attendance, 
amount of sleep prior to examination [17,18], regular 
reading [19,20], and practice questions [21,22] increase 
test performance. There is limited evidence supporting that 
the above behaviors are better practiced with fewer clinical 
work hours. We are unaware of studies demonstrating 
increased reading time resulting from increased work hour 
restrictions and ABSITE scores have been unchanged since 
the implementation of the 80-hour work limit [23,24]. What 
has been shown, however, is exponential SCORE usage 
peaking just prior to the ABSITE, with a significant decline 
upon completion. In theory, if SCORE usage were maintained 
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consistently throughout the year, improvements in ABSITE 
scores and board certification would been achieved. 
Incorporation of SCORE assignments into our ACES program 
has allowed for a more regular usage of the curriculum in 
addition to overall increased time. Gamification of SCORE 
usage should also be considered [25]. 


There are several limitations to our study. This is a single 
institution experience with a small sample size. SCORE 
portal sessions exceeding 180 minutes were excluded as 
these likely represent unattended open sessions. There may, 
however, be residents who genuinely use the curriculum 
for this amount of time. In addition, only the total time 
spent on the SCORE portal was evaluated. The SCORE portal 
includes 800 module topics, TWIS quizzes, chapters from 
multiple textbooks, thousands of multiple-choice questions, 
and narrated operative videos. The number of times the 
portal was opened, the length of individual sessions, the 
topics covered, question bank use, or the amount of time on 
each module was not evaluated. Lastly, residents may use 
alternative resources for ABSITE preparation which is not 
reflected in total SCORE time. 


In conclusion, we have shown that the utilization of the 
SCORE curriculum improves ABSITE scores. Further studies 
are needed to determine exactly which aspects of the portal 
lead to improved scores. This study provides additional 
evidence supporting the importance of the SCORE portal 
in achieving higher ABSITE scores and the fundamental 
knowledge required for surgeons. 
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